A Graph Based Approach to Speaker Retrieval in Talk Show Videos with Transcript-Based Supervision

This paper proposes a graph based strategy to retrieve frames containing the queried speakers in talk show videos. Based on who is speaking and when information from the audio transcript, an initial audio-based step, that restricts the queried person to frames corresponding to when he/she is speaking, with a second step that analyzes visual features of shots is combined. Specifically, based on the production property of talk show video, (1) Shot based graph is constructed first. Then the densest sub-graph is returned as the final result. But instead of direct search (DS) of the densest part, (2) We model the intra node connection and inter node connection by a frame layer degree map to take into account the duration information within each shot node; (3)A graph partition strategy without restriction on the shape and the number of subgraphs is proposed, in which shots containing the same person are more similar to each other. Experiments on one episode of the French talk show “Le Grand Echiquier” show more than 10% improvement to audio only method and more than 7.5% improvement to DS method on average.